RMT 

I M I W | ■ * ■ m Research in 

JL XJL f Middle Level Education 


Micki M. Caskey, Ph.D., Editor 
Portland State University 
Portland, Oregon 


2011 • Volume 34 • Number 10 


ISSN 1940-4476 


Confirming a Middle Grades Leadership Model and Instrument 


Dana L. Bickmore 
Louisiana State University 
Baton Rouge, LA 


Abstract 

The purpose of this replication study was to confirm 
a model of middle grades principal leadership, 
Developmentally Responsive Middle Level Leadership 
(DRMLL), proposed by Brown and Anfara (2002) 
through the validation of the Middle Level Leadership 
Questionnaire (MLLQ), which was developed by 
Anfara, Roney, Smarkola, DuCette, and Gross (2006) 
to measure DRMLL. In this study, the principal 
form of the MLLQ was administered to middle 
grades principals in the State of Georgia. A factor 
analysis indicated the constructs of the principal form 
of the MLLQ support DRMLL, providing further 
confirmation of this model of leadership. However, 
the results of this study indicate principal responses 
and constructs may differ from those of teachers. The 
principal form of the MLLQ may provide an effective 
research tool to examine how antecedents, such as 
professional development, affect principal practice 
and how principal practice affects mediating factors to 
student achievement, such as teacher efficacy. 


Introduction 

The replication of research studies is fundamental 
to developing a body of knowledge that supports 
theoretical constructs and effectively informs practice. 
However, as Hough (2003) noted in his examination 
of middle grades research, “One finding we stumbled 
upon serendipitously (or fortuitously) was that 
virtually no middle level education studies were 
replications of prior efforts” (p. 6). As illuminated by 
the Hough study, principal leadership is an area of 
middle grades literature in need of replication studies. 
Researchers and middle grades proponents have touted 
the importance of the principal in supporting student 
learning (Jackson & Davis, 2000; Leithwood, Louis, 
Anderson, & Wahlstrom, 2004; Marzano, Waters, & 
McNulty, 2005; National Association of Secondary 
School Principals, 2006). Leithwood and colleagues 
(2004) suggested, “Leadership is second only to 
classroom instruction among all school-related factors 
that contribute to what students learn in school” (p. 5). 
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Replicating studies of middle grades principal 
leadership is difficult because of the dearth of studies 
that specifically examine middle grades principalship 
(Brown & Anfara, 2002; Sweetland & Hoy, 2000). 
Particularly lacking are studies focusing on leadership 
practices that middle grades advocates purport as 
supportive of effective middle grades teaching, 
learning, and organizational structures (Brown & 
Anfara, 2002; Valentine, Clark, Hackmannn, & 

Petzko, 2002; Valentine, Clark, Hackmann, & Petzko, 
2004). While general principal leadership research 
and models of effective practice are foundational to 
effective principal leadership, this body of literature 
often fails to account for the uniqueness of school 
context. General school leadership literature asserts 
school context is important (Hoy & Miskel, 2001; Moos, 
Krejsler, & Kofod, 2008), particularly as it relates to 
school climate and culture (Deal & Peterson, 1999; Hoy 
& Hannum, 1997; Sweetland & Hoy, 2000). Models 
of general school leadership, however, provide little 
guidance in the contextual difference in leadership 
at various levels of school configuration. Brown and 
Anfara (2002) and Anfara, Roney, Smarkola, DuCette, 
and Gross (2006) proposed a model of leadership— 
Developmental^ Responsive Middle Level Leadership 
(DRMLL)—that addresses principal leadership in 
the middle grades. This model incorporated student, 
teacher, and organizational components to meet the 
contextual uniqueness of schools that serve young 
adolescents, ages 10-14. The initial research that formed 
the foundation for DRMLL led to the design of an 
instrument, the Middle Level Leadership Questionnaire 
(MLLQ). Anfara and associates (2006) developed the 
instrument to examine principal practice at the middle 
grades level. To date, validation studies of DRMLL as 
a model and MLLQ as an assessment instrument have 
not been conducted. The purpose of this study was to 
validate the MLLQ as an instrument of middle grades 
principal practices and thus support DRMLL as a model 
of middle grades leadership. 

Foundations of Middle Grades Leadership 

The underlying assumption of DRMLL is that 
effective leadership in the middle grades supports the 
tenets, structures, and practices that advocates suggest 
positively affect the learning of young adolescents 
(Anfara, Andrews, Hough, Mertens, Mizelle, & 

White, 2003; Brown & Anfara, 2002; Caskey, 
Andrews, Bishop, Capraro, Roe, & Weiss, 2010; Clark 
& Clark, 2002; Hough, 2003). According to middle 
grades education advocates, effective principals 
should support the general tenets of middle grades 
education through democratic involvement of the 


school and community by developing relationships; 
providing a safe and healthy environment; and 
ensuring that curriculum and instruction are rigorous, 
relevant, challenging, integrative, and exploratory 
(Anfara et al, 2003; Carnegie Council on Adolescent 
Development, 1989; Jackson & Davis, 2000; National 
Forum to Accelerate Middle-Grades Reform, 2004; 
National Middle School Association, 1995, 2010). 
Middle grades organizations also suggest that 
exemplary principals support specific structures and 
practices to achieve the aims of schools educating 
young adolescents (Anfara et al., 2003; National 
Middle School Association, 2010). The structures 
of exemplary middle grades schools include grade 
configuration, heterogeneous class groupings, 
interdisciplinary team organization (teaming), 
flexible/block scheduling, exploratory curriculum, 
and advisory groups (Anfara et al., 2003; George & 
Alexander, 2003; Jackson & Davis, 2000; Kellough & 
Kellough, 2003; National Middle School Association, 
2001, 2003, 2010; Wheelock, 1992). 

Several instruments have been developed for the 
middle grades school context to understand leadership 
and teacher practices (Hoy, Sabo, Barnes, Hannum, 

& Hoffman, 1998; Middle Level Leadership Center, 
2009b). These instruments, however, are primarily 
based on general leadership models with some 
adaptation. Three leadership models undergird these 
instruments and are congruent with exemplary 
middle grades education principles— shared/ 
participatory leadership, instructional leadership, and 
transformational leadership (Anfara, Brown, Mills, 
Hartman, & Mahar, 2000; Brown & Anfara, 2002; 
Clark & Clark, 2000; Hoy & Hannum, 1997; Roney, 
et al., 2004; Scribner, Cockrell, Cockrell, & Valentine, 
1999). While similarities exist among the models, the 
key assumptions and foci vary. 

Shared/participatory leadership is a school leadership 
model that advocates for the participation of teachers, 
sharing of decisions, distribution of leadership, 
and democratic practices across all functions of 
the school (Leithwood & Duke, 1999). The current 
literature purports various titles such as shared 
governance, distributive leadership, collaborative 
leadership, teacher empowerment, and democratic 
leadership, with a recent emphasis on distributive 
and collaborative leadership (Angelle, 2010; Louis, 
Leithwood, Wahlstrom, & Anderson, 2010). 

However, Roney and colleagues (2004) suggested 
that the foundation of each variation is collaborative 
participation in the leadership functions of the school 
by individuals associated with the school. Instructional 
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leadership, as defined by Murphy (1988), is principal 
practice focused on the functions related to the core 
of schools—teaching and learning. Hallinger and 
Murphy (1985) noted instructional leaders (1) defined 
the mission; (2) managed instructional programs, 
which included supervising and evaluating instruction 
and monitoring student progress; and 
(3) promoted a school climate that enforced academic 
standards and promoted professional development. 
Transformational leadership, according to Leithwood 
and Jantzi (1999), “fundamentally aims to foster 
capacity development and higher levels of personal 
commitment to organizational goals on the part 
of the leaders’ colleagues. Increased capacity and 
commitment are assumed to result in extra effort and 
greater commitment” (p. 453). Based on their research, 
Leithwood and Jantzi (1999) proposed six dimensional 
practices of transformational leadership: (a) building 
school vision and goals, (b) providing intellectual 
stimulation, (c) offering individual support, 

(d) symbolizing professional practice and values, 

(e) demonstrating high performance expectations, 
and (f) developing structures to foster participation in 
school decisions. 

Middle Grades Research And General Leadership Models 

There is a small body of research to support 
how shared/participatory, instructional, and 
transformational models of leadership undergird 
principal practice in middle grades schools. The 
National Association of Secondary School Principals 
(NASSP) commissioned a set of three research 
projects in 1981 and 1983, 1992 and 1994, and 2002 
and 2004, to compare a national sample of middle 
grades principals with a subset of 50 to 100 principals 
identified as leading effective middle grades schools 
(Keefe, Clark, Nickerson, & Valentine, 1983; 

Keefe, Valentine, Clark, & Irvin, 1994; Valentine, 

1981; Valentine et al, 2002; Valentine et al, 2004; 
Valentine, Clark, Irvine, Keefe, & Melton, 1993). 

The findings from these sets of descriptive research 
studies indicated effective middle grades principals 
(a) exhibited practices of shared/participatory 
leadership, (b) provided instructional leadership, and 
(c) contributed to the transformation of their schools 
through a vision of exemplary middle grades education. 

In the early set of studies (1981 and 1983), shared/ 
participatory leadership was not a prominent 
characteristic of effective principals. Effective principals 
were, however, more teacher-oriented, with concerns 
for providing teachers enough time for professional 
development. Data from the 1980 set of studies to the 


2000 set of studies revealed that shared/participatory 
leadership was a major theme for effective leadership 
(Petzko, 2004). In the most recent research set, school 
leadership teams appeared in 94% of the effective 
schools, compared to 88% in the national sample. 
Leadership teams and team leaders were more likely 
to be involved in the school improvement process in 
the effective schools. In a review of the 2000 and 2004 
national studies, Petzko (2004) confirmed that effective 
leadership was collaborative, involved shared decision 
making, and included participation of teachers in 
planning for improvement. 

Results from the most recent NASSP national 
leadership studies (Valentine et al, 2004) also indicated 
effective leaders were more likely to value and exhibit 
practices that focused on instruction. Principals in 
effective schools placed greater value on the core of 
education (i.e., teaching and learning) in their schools, 
and they were 22% more likely to involve the entire 
faculty in best practices of exemplary middle grades 
education. In addition, effective schools were engaged 
in professional development more often, and these 
principals involved more of the staff in decisions about 
their own professional development. 

Data from the 2000 and 2002 set of NASSP studies 
(Valentine et al, 2002) also suggested effective middle 
grades principals were more likely to exemplify 
transformational leadership practices. Effective middle 
grades schools were more likely to have a school vision 
and mission and were more likely to have comprehensive 
school improvement plans. Effective principals at these 
schools facilitated a greater level of implementation of 
the restructuring elements of exemplary middle grades 
education, such as higher levels of interdisciplinary 
teaming and advisory programs. 

Anfara and colleagues (2000) and Brown and Anfara 
(2002) provided additional empirical evidence that 
effective middle grades leadership is associated with 
shared/participatory, instructional, and transformational 
leadership models. Anfara and colleagues (2000) 
identified five themes from interviews and surveys 
that had asked principals to define success. All themes 
supported the importance of shared/participatory 
leadership. First, effective middle grades principals had 
a positive outlook and were satisfied with their jobs, 
including providing encouragement and motivation to 
staff and students. Second, effective principals were 
teacher-oriented, providing teachers with common time 
to collaborate and plan and expecting teachers to be 
student-centered. Third, principals were supportive of 
parent and community involvement in their schools. 
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Fourth, effective principals tolerated ambiguity. 

Fifth, effective middle grades principals assembled, 
developed, and maintained a staff dedicated to middle 
grades education. The last theme, developing staff, also 
connoted the importance of professional development, 
an integral aspect of instructional and transformational 
leadership models. 

In a reanalysis of the same data set, Brown and 
Anfara (2002) focused on the transformational 
leadership skills of middle grades principals. They 
examined the strategies used by these middle grades 
principals before implementing reform initiatives 
and during the process of transformation. From the 
data, the researchers gleaned principals’ effective 
practices. Middle grades principals saw themselves as 
“accepting] their role as catalyst and as vision keeper” 
(p. 21). Brown and Anfara found effective principals 
recognized that school transformation required 
a change in school culture. Effective principals 
involved others in the process of culture building 
through democratic practices and shared decision 
making. Collaborating, building trust and consensus, 
training, and communicating were also found to be 
necessary to change norms and values, and effective 
principals incorporated these practices into their 
repertoire. This research supported the practices of 
the transformational leadership model as important in 
effective middle level leadership. Brown and Anfara’s 
research also found shared/participatory leadership to 
be congruent with effective leadership that includes 
continual discussion, collaboration, democratic 
practices, and shared decision making. 

Several examples of research instruments, based 
on these general models, examine middle grades 
leadership. For example, the Middle Level Leadership 
Center at the University of Missouri provides surveys 
for use by middle grades leaders, including the Audit 
of Principal Effectiveness (Middle Level Leadership 
Center, 2009a) and Instructional Practices Survey 
(Middle Level Leadership Center, 2009b). These 
surveys are adaptations of the transformational 
leadership model (Valentine et al., 2001). Hoy and 
Hannum’s (1997) Organizational Health Inventory for 
Middle Schools, based primarily on the instructional 
leadership model, is another example of an instrument 
used to examine middle grades leadership. 

An Integrated Model of Middle Level Leadership 

According to Brown and Anfara (2002), shared/ 
participatory, instructional, and transformational 
leadership models and the instruments derived from 


these models were foundational but insufficient 
in guiding middle grades principal practice. They 
lacked the important contextual tenants, structures, 
and practices of middle grades education. Based 
on finding from their study of 44 principals in 
Pennsylvania, New Jersey, and North Carolina, 
and their involvement in previous research related 
to middle grades principals (Anfara et al., 2000; 
Brown & Anfara, 2003), Brown and Anfara proposed 
a model of leadership titled Developmentally 
Responsive Middle Level Leadership (DRMLL). 
DRMLL is three-dimensional: “(1) responsiveness to 
the developmental needs of middle grades students; 
(2) responsiveness to the developmental needs of 
faculty who support learning for middle grades 
students; and (3) responsiveness to the development 
of the middle school itself as a unique, innovative 
entity” (Brown & Anfara, 2002, p. 149). Each 
dimension included structures and practices of 
exemplary middle grades education, such as building 
a culture of community and a focus on curriculum, 
instruction, and assessment. Finally, the authors 
listed specific middle grades principal practices 
for each dimension. For the “responsiveness to 
students” dimension, principals must understand the 
intellectual, physical, psychological, social, moral, 
and ethical characteristics of young adolescents. 
Principals must also believe that all students 
can succeed. Understanding the need to connect 
educational administration to teaching and learning, 
and governing democratically and collaboratively are 
examples of practices presented for the “responsive 
to faculty” dimension. For the “responsive to the 
needs of the school” dimension included a knowledge 
and implementation of the structures and practices 
of exemplary middle grades education. In addition, 
principals must act as catalysts for change. DRMLL 
incorporated the three leadership models used 
as the basis for effective leadership outlined by 
middle grades advocates—shared/participatory, 
instructional, and transformational. DRMLL moved 
beyond these models to provide the contextual 
nuances of middle grades principal leadership not 
found in any one general leadership model. 

Anfara and associates (2006), in an attempt to test 
and operationalize DRMLL as a model of leadership, 
developed a survey instrument, the Middle Level 
Leadership Questionnaire (MLLQ). The purpose of 
the MLLQ was to evaluate the behaviors of middle 
grades principals. The instrument consisted of 
questions related to the demographic characteristics 
of the participants and an inventory of middle school 
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practices. Anfara and associates (2006) and Roney 
and colleagues (2004) established content validity of 
the principal behavior portion of the MLLQ through 
a panel of 45 experts composed of middle level 
principals and middle level authorities. This group 
provided feedback on the original 65-item survey by 
indicating the importance and clarity of questions. 
Following a statistical analysis of the rankings 
generated by the panel, the MLLQ was reduced to 35 
items using a five point Likert scale. Two forms of the 
final 35-item questionnaire were developed, one for 
principals (Form A) and the other for teachers 
(Form B). The two forms mirror each other with 35 
items related to principal behaviors and an inventory of 
middle school practices. The MLLQ was administered 
to nine principals and the teachers in their schools. 

The response rate of teachers was not reported. A 
principal axis factor analysis with a Varimax rotation 
was conducted on both the principal and teacher 
questionnaire to establish construct validity. This 
analysis yielded five factors: (1) developmentally 
appropriate learning environments/support of 
teachers; (2) best practices; (3) developmentally 
appropriate learning environment/support of student 
needs; (4) student self-confidence and competence; 
and (5) responsiveness to student needs/support of 
teachers. Two items did not fit the factor construct, 
resulting in a 33-item instrument. Reliability was 
determined through measures of internal consistency 
(alpha coefficients) of items within each factor. The 
researchers reported alpha coefficients of .93, .89, .81, 
.76, and .72 for the five factors, respectively. 

Roney and colleagues (2004) suggested that the 
five factors fit into the DRMLL model, with factor 
one and five corresponding to responsiveness to 
school needs, and factors three and four relating 
to responsiveness to students, while factor two 
remained associated with responsiveness to teachers. 
Flowever, there were significant difference in 
responses between principals and teachers when 
t-tests were conducted comparing the two groups. 
Statistically significant differences were found 
between principals’ and teachers’ responses related 
to factor one, developmentally appropriate learning 
environment/support of teachers, and factor five, 
responsiveness to student needs/support of teachers. 
The limited number of principal responses and the 
variation in responses within constructs between 
principals and teachers suggests that replication and 
further examination of the MLLQ is warranted. 


Methods 

The purpose of this study was to confirm the 
construct validity and reliability of Part 11 of Form 
A of the MLLQ, which measured principals’ 
perceptions of their own middle grades leadership 
behaviors. Confirming the validity and reliability 
of the principals’ form of the MLLQ could provide 
practitioners with confidence that the instrument 
might support their efforts to improve practice. 
Validation of Form A could also provide an 
instrument for future research related to middle 
grades principals’ perceptions of their leadership. 
Finally, confirmation of the validity of the MLLQ 
provides further support for a model of leadership 
specific to the context of the middle grades. If 
used in preparation programs, in conjunction with 
general leadership models, this model might enhance 
professional practice of aspiring middle grades 
principals. For these reasons, an exploratory factor 
analysis was used to establish factors for principal 
perceptions and for reliability using only Form A. 

After the authors of the MLLQ granted their 
permission, the MLLQ was mailed to the entire 
population (N= 393) of full-time principals of middle 
grades schools in the state of Georgia 1 For the purposes 
of this study, middle grades schools were defined as 
schools, or portions of schools, containing grades 6, 7, 
and 8 or 7 and 8. The population, rather than a sample, 
was selected to provide sufficient numbers of returned 
responses for correlational analyses. 

Correlational statistical procedures, including factor 
analysis and scale reliability testing, were used on 
the returned questionnaires to establish construct 
validity and internal reliability of the adapted MLLQ. 
Specifically, an exploratory factor analysis was 
conducted using a principle component analysis to 
determine factors for the adapted MLLQ. Factors 
were extracted and rotated using a Varimax rotation. 
Alpha coefficients for each factor determined through 
the factor analysis were used to establish construct 
validity and internal reliability of the adapted MLLQ. 

Results 

The response rate for 393 mailed surveys was 42.5%. 
Although 168 surveys were returned, one survey 
was not usable because a significant portion of the 
data was missing. It should be noted that missing 
data on individual items did occur. For example, 
some surveys were missing demographic data or data 
on an item of the adapted MLLQ. Such omissions 
constituted less than 5% of any respondent’s total 
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responses. Although there is no definitive, agreed 
upon single standard for acceptable response rates 
to questionnaires, methodologists suggest rates of 
50% to 75% may be appropriate (Cohen, Manion, 

& Morrison, 2000; Fowler, 2002; Rea & Parker, 
1997). When response rates fall below these levels, 
as is the case in this study, questions arise as to 
how well the results generalize to the population. 
Methodologists suggest when response rates may be 
in question, researchers can minimize the difference 
between respondents and non-respondents if they 
can ascertain the differences that might exist 
between the two groups, such as age, experience, or 
socioeconomic level (Fluck, 2000). The researcher in 
this study compared several demographic factors of 
respondents in this study with the general population 
of middle grades principals reported by the Georgia 
Department of Education (GDOE) (2005). Means 
were compared with respect to gender, ethnicity, 
and the free and reduced-price lunch status of the 
schools in which principals worked. The percentages 
of males and females among respondents in this 
study were 46% and 52%, respectively, with 1.8% 
of the principals not responding to this item. This 
compared to the population percentages of 47% 
and 53%, respectively. The ethnic backgrounds 
reported by middle grades principals in this study 
were 70% white and 27% African American, 
while the population ethnicity reported by the 
GDOE was 65% and 34%, respectively. The mean 
for free and reduced-price lunches at schools in 
which the principals worked was reported as 53% 
compared to the population percentage of 50%. 

No other comparable demographic data were 
available through the GDOE. Flowever, the types of 
communities reported in this study were comparable 
to those reported in a dissertation study conducted 
by Warren (2002). In the Warren study, 17.4% of 
the middle grades principals reported working in 
urban schools, 37.5% reported working in suburban 
schools, and 45.1% reported working in rural schools. 
Similarly, respondents in the present study reported 
a distribution of 15.6% urban, 37.1% suburban, and 
43.7% rural. A reasonable correspondence between 
the response sample and the general population 
appears to exist. Although the sample of respondents 
in this study tended to report slightly less diversity 
than the population, the other demographic factors 
were very similar. Based on these comparisons, it 
appears reasonable to assert a cautious generalization 
of the results of this study to the general population 
of middle grades principals in the state of Georgia. 


While the principals in this study appeared to 
represent the middle school principals in Georgia, 
they did not represent those of the national middle 
school population. According to national research 
studies conducted by Valentine and associates (2002), 
principals in this study were much more likely to 
have a master’s degree and specialist degree in 
middle level education (34%, 30%) than those of 
the national study (6%, 4%). In this study, 27% of 
principals were African American compared to only 
5% in the national study. In addition, the majority of 
principals surveyed in this study were female (52%), 
while only 27% of the principals were female in 
Valentine and associates’ (2002) study. 

Factor Analysis 

An initial factor extraction of the useable responses 
from the mailed MLLQ, using Statistical Package 
for the Social Sciences (SPSS) version 12.0, was 
conducted. The sample size for this analysis, 167, 
met the subjects-to-variables ratio criteria, which 
should be not less than five participants per item on 
the MLLQ, or a minimum of 165 participants (Bryant 
& Yarnold, 1995). Kaiser’s criterion (1960), which 
SPSS uses by default, was applied. SPSS extracted 
nine factors with eigenvalues greater than 1. The first 
factor accounted for 35.57% of the variance, with the 
remaining eight factors accounting for an additional 
33.92% of the variance, for 69.49% of the variance. 
Cattell’s (1977) scree test was also performed to 
determine which factors to retain. Based on the data 
curve presented in the scree plot, two possible factor 
models were indicated—either a 3-factor or 5-factor 
model was justified. According to Green and Salkind 
(2005), a third criterion beyond eigenvalues and 
scree tests should be applied to the choice of factors 
to extract—the a priori conceptual constructs that 
may underlie the factors. Considering the 5-factor 
model outlined by Anfara and associates (2006) in the 
original testing of the MLLQ, five factors comprised 
a possible model solution. Alternately, the theoretical 
framework that supported the original MLLQ, 
Developmentally Responsive Middle Level Leadership 
(DRMLL), suggested three factors as a possible model 
solution corresponding to responsiveness to students, 
responsiveness to faculty, and responsiveness to 
exemplary middle grades education in schools, as it 
supports faculty and students. 

With the scree plot and the a priori conceptual 
frameworks in mind, the researcher conducted both a 
5-principal (k = 5) and 3-principal (k = 3) component 
analysis using a Varimax rotation with Kaiser 
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normalization. The component matrix for the 5-factor 
and 3-factor models, are displayed in Tables 1 and 2, 
respectively. The 3-factor model provided a better fit 
than the 5-factor model for three reasons. First, the 
3-factor model provided a more equal distribution 
of variance among the factors. Second, the 3-factor 
model resulted in a more equal distribution of question 
items to be loaded on each factor. The third and most 
important reason for choosing the 3-factor model was 
its fit with the constructs of DRMLL. The 5-factor 
model matched neither the MLLQ factors presented 
by Anfara and associates (2006) nor the underlying 
constructs of the DRMLL model. The items of the 
3-factor model grouped closely with the three constructs 
of the DRMLL theoretical model: developmental 
responsiveness of leaders to students, faculty, and 
exemplary middle grades education in schools as it 
supports faculty and students (see Table 3). 

The minimum factor-loading criterion for retaining 
items in a factor model after rotation is .30 
(Nunnally, 1978). Question 2, “As the principal of 
a middle school 1 promote the caring relationships 
between teachers, staff, and students through 
structures like advisory period etc.,” did not load 
above a .30 on any of the factors in the 3-factor 
model. This item was eliminated from the model, and 
a final Varimax rotation of the 3-factor solution was 
conducted. The total variance explained by the model 
after rotation and without question 2 was 47.76%. 

Reliability of the model was determined through 
internal consistency as measured by Cronbach’s alpha 
coefficients of items within each factor. The alpha 
coefficients for factors 1 through 3 were .91, .87, and 
.80, respectively. Methodologists consider reliability 
correlation coefficients of .70 or greater adequate to 
establish reasonable confidence in the reliability of the 
instrument (De Vaus, 2002; Gay & Airasian, 2003). 

Discussion 

The results of this replication study indicate both 
similar and alternate conclusions from those 
made by Brown and Anfara (2002), Anfara and 
associates (2006), and Roney and colleagues (2004) 
in relationship to the MLLQ and the underlying 
model of middle grades leadership, DRMLL. 

Form A of the MLLQ, as adapted for this study 
(heretofore labeled the Adapted MLLQ), appears to 
be a valid and reliable instrument to assess principal 
perceptions of their own behaviors related to middle 
grades education. Flowever, with a larger sample 
of principals in this study than the original, the 


constructs that support the instrument are different 
from those found by Anfara and associates. The 
constructs that support the Adapted MLLQ in this 
study correspond directly to the model of leadership 
that undergirds the instrument, suggesting that the 
DRMLL is a model of leadership appropriate for 
framing middle grades principal leadership and 
informing principal practice in the middle grades. 
Specifically, middle grades principal leadership as 
outlined by responsiveness to the developmental 
needs of (a) middle grades students, (b) faculty 
who support middle grades students, and (c) middle 
grades schools as they support students, may provide 
a lens to guide effective practice through research 
and principal reflection. 

The variations in results from this study and those of 
Anfara and associates (2006) can be attributed to the 
numbers of participants in each study. The original 
MLLQ consisted of 33 questions, which, according 
to Bryant and Yarnold (1995), would require 165 
completed questionnaires to deal sufficiently with 
the subjects-to-variables ratio criteria. The limited 
number of principal responses (nine in the original 
study) may not have been sufficient to develop 
the constructs for the MLLQ in relationship to 
principals’ perceptions of their own behaviors. 

This variation highlights three issues related to 
the use of the MLLQ and the underlying DRMLL 
by various groups associated with middle grades 
education—practitioners, researchers, and those 
preparing educational leaders. Anfara and associates 
(2006) suggested that practicing principals use 
the MLLQ to improve practice. Principals can 
accomplish this by comparing their own perceptions 
of practice to faculty perceptions of their leadership 
behaviors. Anfara and associates have developed 
a scoring guide, combining teacher and principal 
responses to the MLLQ. Because the MLLQ form 
B was administered to an adequate number of 
teachers to satisfy the subjects-to-variables ratio 
criteria, it may provide the principal with valid and 
reliable information about his or her behaviors as a 
middle grades principal as perceived by the school 
faculty. However, if principals use the instrument 
to compare their own self-perceptions with staff 
perceptions of middle grades leadership, there may 
be difficulties with validity and reliability of the 
instrument, at least if the constructs are directly 
compared between principal and teacher results, 
as advocated by Anfara and associates. Valid and 
reliable instruments are critical. If principals invest 
the time to administer, tabulate, score, and compare 
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results to scoring tables, they should be confident the 
results provide them with accurate information that 
can be used to improve practice. Further research is 
needed to confirm the relationship between teacher 
and principal constructs of the MLLQ and how these 
constructs can be used to improve practice. Such 
research may determine if information obtained from 
comparing teacher and principal perceptions will 
accurately inform middle grades principals. 

Alternately, the Adapted MLLQ may be an important 
instrument for comparing middle grades principal 
practice with outcome measures of effectiveness. 
Eventually, middle grades principal behaviors that 
are congruent with middle grades practices may be 
compared to student achievement data. However, with 
the indirect nature of principal behaviors on student 
outcomes, the results of the Adapted MLLQ may be 
used to compare middle grades principal practice 
with mediating factors that have been identified as 
influencing student achievement, such as teacher 
efficacy. The Adapted MLLQ may also be used to 
compare antecedents that influence principal practice, 
such as experience and preparation, or factors that may 
affect practice, such as professional development. 

What distinguishes the Adapted MLLQ from other 
instruments developed to measure effective principal 
practice is its focus on the context of middle grades 
leadership and the developmental nature of middle grades 
students, teachers, and schools. Other instruments based 
on various general models of leadership, such as those 
developed by Heck, Larsen, and Marcoulides (1990) in 
relationship to instructional leadership, have provided 
important information about effective leadership by using 
the instrument in comparison to various antecedent, 
mediating, and outcome factors. There have also been 
attempts to develop instruments to be used by middle 
grades researchers and principals, such as the those 
developed by the Middle Level Leadership Center 
(MLLC) (2009a), that are based on general leadership 
models. However, the Adapted MLLQ is a valid and 
reliable instrument that supports a middle level leadership 
model and provides opportunities to further explicate the 
important nuances of leadership in the middle grades. 

The notion of a model of leadership directly associated 
with the tenets and practices of middle grades 
leadership, supported by a valid instrument, also has the 
potential to enhance the education of aspiring principals. 
The vast majority of principals in the United States are 
K-12 certified, with only five states requiring a middle 
level credential (Gaskill, 2002). Beyond generic models 
of school leadership, the current Interstate School 


Leaders Licensure Consortium (ISLLC) Standards 
(Council of Chief State School Officers, 2008) 
governing principal preparation are designed to apply 
to all school levels. The use of DRMLL as a model of 
leadership, supported by evidence of its validity, may 
provide those preparing school leaders with a tool to 
interject effective middle grades leadership practices 
into instruction in tandem with more general models 
and standards of preparation. 

While this study provided evidence of a valid and 
reliable instrument that further confirms a middle 
grades leadership model, the investigation has several 
limitations. The greatest limitation is the question this 
study poses to the congruence of the MLLQ related 
to the two forms of the instrument, the principal form 
and the teacher form. This investigation did not attempt 
to replicate a study that could determine the construct 
validity or reliability of the MLLQ from the teacher’s 
perspective. Without further study, the Adapted MLLQ 
and the MLLQ must be considered two instruments 
measuring related but different constructs. The Adapted 
MLLQ constructs included those of DRMLL— 
responsiveness to the developmental needs of middle 
grades students, faculty who support middle grades 
students, and middle schools as a unique innovative 
entity. The MLLQ (teacher form) constructs were 
identified as—developmental^ appropriate learning 
environments/support of teachers, best practices, 
developmentally appropriate learning environment/ 
support of student needs, promotion of student self- 
confidence and competence, and responsiveness to 
student needs/support of teachers. Further studies 
need to be conducted to test the MLLQ with both 
principals and their faculties in large enough numbers 
to validate both forms of the instrument to determine 
if the differences in constructs are significant, both 
statistically and practically. 

A second limitation of this study is the use of principal 
responses from one state in the United States. Just 
as there are contextual differences in leadership at 
various grade levels, there are differences between 
states, both in leadership requirements and in middle 
grades practices. As noted earlier, the principals in this 
study were dramatically different from those in the 
Valentine and associates’ national study (2002). These 
demographic deviations, as previously stated, suggest 
studies with geographically more diverse participants 
are needed to substantiate this instrument. 
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Table 1 

Rotated Component Matrix for 3-Factor Model 


Factors 

Question 

1 


2 

3 

QJ 

.111 


.177 

.358 

Q_2 

-.061 


.238 

.214 

Q_3 

.103 


.256 

.418 

Q_4 

.141 


.196 

.509 

Q_5 

.164 


.397 

.515 

Q_6 

.339 


.186 

.550 

Q_7 

.237 


-.009 

.646 

Q_8 

.275 


.092 

.644 

Q_9 

.178 


.315 

.576 

Q_10 

.615 


.230 

.346 

Q_n 

.258 


.507 

.273 

Q_12 

.319 


.404 

.225 

Q_13 

.537 


.272 

.433 

Q_14 

.521 


.363 

.321 

Q_15 

-.089 


.090 

.410 

Q_16 

.223 


.707 

.291 

Q_17 

.308 


.549 

.292 

Q_18 

.330 


.531 

.080 

Q_19 

.460 


.610 

.162 

Q_20 

.276 


.706 

.073 

Q_21 

.197 


.706 

.200 

Q_22 

.717 


.277 

.187 

Q_23 

.452 


.634 

.022 

Q_24 

.580 


.297 

.240 

Q_25 

.734 


.212 

.059 

Q_26 

.783 


.303 

.094 

Q_27 

.749 


.213 

.114 

Q_28 

.695 


.099 

.232 

Q_29 

.424 


.371 

.193 

Q_30 

.285 


.089 

.637 

Q_31 

.423 


-.096 

.511 

Q_32 

.476 


.224 

.160 

Q_33 

.661 


.314 

.218 


Extraction Method: Principal Component Analysis. 
Rotation Method: Varimax with Kaiser Normalization. 
Rotation converged in 8 iterations. 
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Table 2 

Rotated Component Matrix for 5-Factor Model 


Factors 

Question 

1 

2 

3 

4 

5 

Q_i 

.042 

.163 

.347 

.358 

-.102 

Q_2 

.036 

.143 

-.128 

.274 

.476 

Q_3 

.132 

.139 

.111 

.834 

.042 

Q_4 

.180 

.080 

.174 

.769 

.188 

Q_5 

.128 

.357 

.364 

.359 

.244 

Q_6 

.257 

.205 

.562 

.115 

.187 

Q_7 

.125 

.011 

.694 

.218 

.040 

Q_8 

.173 

.111 

.668 

.167 

.142 

Q_9 

.065 

.354 

.629 

-.030 

.239 

Q_10 

.525 

.281 

.473 

.039 

.009 

Q_ii 

.180 

.512 

.295 

.245 

-.045 

Q_12 

.233 

.427 

.306 

.189 

-.122 

Q_13 

.395 

.342 

.631 

.041 

-.091 

Q_14 

.380 

.452 

.557 

-.131 

-.051 

Q_15 

-.044 

.052 

.179 

-.090 

.660 

Q_16 

.148 

.733 

.306 

-.039 

.234 

Q_17 

.249 

.563 

.290 

.069 

.169 

Q_18 

.278 

.532 

.110 

.261 

-.143 

Q_19 

.407 

.617 

.179 

.219 

-.021 

Q_20 

.249 

.717 

.055 

-.019 

.190 

Q_21 

.219 

.664 

.008 

.163 

.385 

Q_22 

.686 

.295 

.224 

.172 

.003 

Q_23 

.405 

.653 

.076 

.117 

-.057 

Q_24 

.546 

.323 

.267 

.018 

.158 

Q_25 

.737 

.217 

.072 

.195 

-.031 

Q_26 

.778 

.325 

.123 

.053 

.095 

Q_27 

.759 

.221 

.104 

.121 

.098 

Q_28 

.653 

.136 

.316 

.040 

.034 

Q_29 

.410 

.376 

.161 

.049 

.199 

Q_30 

.289 

.057 

.440 

.133 

.562 

Q_31 

.440 

-.122 

.360 

.163 

.402 

Q_32 

.532 

.204 

.017 

.010 

.408 

Q_33 

.610 

.339 

.281 

.158 

-.024 


Extraction Method: Principal Component Analysis. 
Rotation Method: Varimax with Kaiser Normalization. 
Rotation converged in 7 iterations. 
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Table 3 

DRMLL Constructs and Adapted MLLQ Questions Associated with Constructs 


Factor 

Factor 1: Faculty 
Factor 1: Faculty 

Factor 1: Faculty 

Factor 1: Faculty 
Factor 1: Faculty 

Factor 1: Faculty 

Factor 1: Faculty 
Factor 1: Faculty 
Factor 1: Faculty 
Factor 1: Faculty 
Factor 1: Faculty 

Factor 1: Faculty 

Factor 2: Students 

Factor 2: Students 
Factor 2: Students 

Factor 2: Students 
Factor 2: Students 
Factor 2: Students 

Factor 2: Students 

Factor 2: Students 

Factor 2: Students 


Question: As the principal of a middle school, I... 


10. have a vision of what an exemplary middle school is and strive to bring that vision to life. 

13. demonstrate an understanding of the intellectual, physical, psychological, 
and social characteristics of young adolescents. 

14. demonstrate an understanding of the relationship between the cognitive and affective 
needs of young adolescents. 

22. make decisions based on young adolescent development and effective middle level practices. 

24. provide time for general education teachers to collaborate with special education 
teachers to meet the diverse needs of young adolescents. 

25. provide teachers time, grouping, and instructional strategies to help individual 
students to achieve mastery of subject matter. 

26. encourage teachers in their efforts to respond to the needs of young adolescents. 

27. encourage teachers in their use of a wide variety of instructional approaches and materials. 

28. encourage active discovery learning on the part of students rather than teacher lectures. 

29. encourage activities such as special-interest classes and hands-on learning. 

32. encourage teachers to make connections across disciplines to reinforce 
important concepts. 

33. require teachers to provide classroom activities that address the needs of 
academically diverse learners who vary greatly in readiness, interests, and 
learning profiles. 

11. provide curricular materials that enhance young adolescents’ acceptance of self and 
others and that enable them to accept differences and similarities among people. 

12. provide adequate counseling/advisory opportunities. 

16. provide students with opportunities to explore a rich variety of topics to develop 
their identity and demonstrate their competence. 

17. develop connections with and involve families in the education of their children. 

18. provide age-appropriate, co-curricular (or extra-curricular) activities. 

19. provide students with opportunities to explore, make mistakes, and grow in a safe, 
caring environment. 

20. encourage mature value systems by providing opportunities for students to examine 
options of behavior and to study consequences of various actions. 

21. regard young adolescents as resources in planning and program development and involve 
them in meaningful roles. 

23. allow teachers and students to plan activities that integrate genders. 


Continued on page 12 
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Table 3 (continued) 

DRMLL Constructs and Adapted MLLQ Questions Associated with Constructs 


Factor 


Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 

Factor 3 

: School 


Question: As the principal of a middle school, I... 

1. design and implement policies and procedures that reflect the needs of young adolescents. 

3. provide transition programs from middle to high school for my middle school students. 

4. provide transition programs from elementary to middle school for my middle 
school students. 

5. organize the curriculum around real-life concepts. 

6. advocate for middle schools and the middle school concept in the school district. 

7. prepare a daily schedule that includes time for team planning and meeting. 

8. stay current on what the research says about best practices for middle schools. 

9. group students and teachers in small learning communities. 

15. spend time each day with students. 

30. create opportunities for professional development for teacher/staff that address 
strategies for meeting the needs of young adolescents. 

31. support appropriate instructional strategies with the necessary resources 
(i.e., money, time needed, etc.). 


Conclusion 

Principals in most states are prepared and receive 
certifications for grades K-12 (Gaskill, 2002). Current 
models of leadership are also based on general 
principles of leadership applicable to principals 
guiding K-12 schools. Yet students and school contexts 
vary depending on the grade configuration of the 
school. This study contributed to validating a model of 
leadership that outlined principal behaviors congruent 
with effective middle grades education through testing 
an instrument based on that model. Results of this 
study also support the confirmation of the Adapted 
MLLQ as a measure of principal perceptions of their 
middle grades behaviors that are developmentally 
responsive to middle grades students, teachers, and 
the middle grades school. While this study provided 
support that the DRMLL may be an efficacious 
framework for examining middle grades principal 
practices, it did not confirm earlier research that the 
original principal form of the MLLQ is a 5-factor 
instrument, with the same constructs that undergird 
the teacher version of the instrument. Further studies 
are needed to explicate whether the Adapted MLLQ is 
two instruments—one measuring the three DRMLL 
constructs for principals and one measuring a 
compatible variation of the model for teachers based 
on five constructs. 


Disaffirmation of the congruence of the teacher and 
principal forms of the MLLQ calls into question 
the use of the instrument by practicing principals 
who wish to compare the perceptions of their own 
middle grades leadership practices with those of their 
staff. Practitioners should have valid and reliable 
measures to support their efforts to improve practice. 
As suggested by Hough (2003), echoed by Anfara 
and associates (2003), and confirmed by this study, 
replication of research studies associated with middle 
grades education is important in providing valid 
information that practitioners can use, with confidence, 
to improve practice and support student outcomes. 

The Adapted MLLQ, however, appears to be an 
instrument that can be used to examine middle 
grades principal perceptions of their own behaviors 
related to student outcomes or to mediated factors 
leading to student outcomes, such as teacher 
efficacy. In addition, the instrument may be a 
means to measure antecedents that affect middle 
grades principal practice, such as preparation and 
professional development. This study provided 
added support for the development of a leadership 
model, and an instrument supported by that model. 
Further confirmation of the DRMLL as a model of 
middle grades leadership may also support those 
preparing middle grades leaders through providing 
an additional model of leadership that specifically 
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highlights middle grades tenets, structures, and 
practices. With the limited research examining 
middle grades principal behaviors in relation to 
exemplary middle grades education, this study may 
advance our understanding of effective principal 
leadership for those working in the middle. 
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